RNN-Test: Towards Adversarial Testing for Recurrent Neural Network Systems

While massive efforts have been investigated in adversarial testing of convolutional neural networks (CNN), for recurrent (RNN) is still limited and leaves threats vast sequential application domains. In this paper, we propose an framework RNN-Test RNN systems, focusing on sequence-to-sequence (seq2seq) tasks widespread deployments, not only classification First, design a novel search methodology customized models by maximizing the inconsistency states against their inner dependencies to produce inputs. Next, introduce two state-based coverage metrics according distinctive structure RNNs exercise more system behaviors. Finally, solves joint optimization problem maximize state coverage, crafts inputs various different kinds For evaluations, apply four common structures. On tested models, approach demonstrated be competitive generating inputs, outperforming FGSM-based DLFuzz-based methods reduce model performance sharply with 2.78% 37.94% higher success (or generation) rate. could also achieve 52.65% 66.45% adversary rate than testRNN MNIST LSTM model, as well 53.76% 58.02% perplexity 16% generation DeepStellar PTB language model.Compared traditional neuron proposed guidance excel 4.17% 97.22%